1. Field of the Invention
The present invention relates to an image processor for applying correction (such as black level correction and white balance correction) which are typically required in an image sensor including a plurality of channels for reading out image data (“image data readout channels”).
2. Description of the Background Art
FIG. 23 is a block diagram schematically illustrating a configuration of an image processing LSI for a digital (still or video) camera according to the conventional arts. In the conventional image processing LSI illustrated in FIG. 23, only a single channel RCH0 for reading out image data is provided in an image sensor PIS, so that respective items of image data in a plurality of pixels are sequentially subjected to raster scanning to be read out. Meanwhile, an array of color filters (an order in which colors are put) respectively provided on a plurality of pixels placed in an effective pixel region of the image sensor PIS is in a so-called RGB Bayer pattern.
Each of the items of image data which are sequentially read out from the image sensor PIS in accordance with pixel clocks (not illustrated) is subjected to A/D conversion in an A/D converter 300P, first, and then is input to an image sensor data pre-processing unit (which will be hereinafter referred to as an “SPU” (abbreviation of “sensor processing unit”)) 100P which is placed on a C-MOS LSI. The SPU 100P is not a circuit for performing substantial image processing, but a unit for performing processes which are required due to problems typical of the image sensor PIS, that is, black level correction, while balance correction, and defective pixel correction, on the image data input from the image sensor PIS, to normalize a signal provided by each of the pixels with color filters which are arranged in an RGB Bayer pattern (which signal will hereinafter be referred to as an “RGB Bayer signal”).
FIGS. 24A, 24B, and 24C schematically illustrate normalization of an RGB Bayer signal input from the image sensor PIS. FIGS. 24A, 24B, and 24C will be adopted in a first preferred embodiment of the present invention which will be later described. A relationship between an output value and brightness of each of color signals of a red (R) color signal, a green (G) signal, and a blue (B) signal shown in FIG. 24A is assumed to be linear, and the respective color signals are subjected to black level correction, so that respective original points thereof are aligned to one another (refer to FIG. 24B). Further, respective color signals are subjected to white balance correction, so that the color signals are normalized with respective characteristic lines thereof overlapping one another (refer to FIG. 24C).
The normalized RGB Bayer signal is then input to an RPU 200P, where substantial image processing such as gamma correction is performed on the normalized RGB Bayer signal.
FIG. 25 is a block diagram schematically illustrating a configuration of a principal portion of the SPU 100P according to the conventional arts. Referring to FIG. 25, a global H counter 1 is a global horizontal counter which receives pixel clocks CLK as input signals and counts the number of times of reception of the input signals to generate a count value HCNT, to thereby make or control a timing for horizontally reading out image data supplied from the image sensor PIS. Also, a global V counter 2 is a global vertical counter which counts the number of times of reception of a reset signal output from a carry-out terminal CO of the global H counter 1 to provide a count value VCNT and increments the count value VCNT one by one each time the reset signal is received in synchronization with the pixel clocks CLK, to thereby make or control a timing for vertically reading out image data supplied from the image sensor PIS. It is additionally noted that both the counters 1 and 2 are elements which are also employed in the first preferred embodiment of the present invention which will be later described. For the SPU 100P, it is sufficient that a 2×2-size repeat block of a timing signal for reading out correction data is provided in a color filter array in an RGB Bayer pattern (which will hereinafter be referred to an “RGB Bayer array”) because the image sensor PIS includes only a single readout channel. Accordingly, combining the least significant bit HCNT(0) of the count value HCNT of the global H counter 1 and the least significant bit VCNT(0) of the count value VCNT of the global V counter 2 results in generation of a 2-bit color selection timing signal COLSEL. Also, a black level correction value register BLR storing a correction value used for black level correction is previously provided in the SPU 100P. The black level correction value register BLR includes registers BL0, BL1, BL2, and BL3 which store respective black level correction values at four locations in the above-mentioned 2×2-size repeat block to which respective colors (four colors in all) in an RGB Bayer array are allocated. More specifically, the register BL0 stores a black level correction value for R color which is allocated to a location “00” in the repeat block. The register BL1 stores a black level correction value for Gr color which is allocated to a location “01” in the repeat block. The register BL2 stores a black level correction value for Gb color is allocated to a location “10” in the repeat block. The register BL3 stores a black level correction value for B color which is allocated to a location “11” in the repeat block. Likewise, a white balance correction value register WBR storing a correction value used for white balance correction is provided in the SPU 100P. Regarding the SPU 100P, it is particularly notable that both a black level correction value and a white balance correction value at each location and for each color are selected directly by the color selection timing signal COLSEL via selectors 11P and 12P, respectively, as illustrated in FIG. 25. Additionally, reference numerals 13P and 14P in FIG. 25 denote a black level corrector and a white balance corrector, respectively.
<First Problem>
In an effort to generate a higher definition image, there is an increasing demand for an increase in resolution of an image sensor (such as a CCD sensor or a CMOS sensor), that is, an increase in the number of pixels in an image sensor. In order to satisfy the foregoing demand, an image sensor including 8M or more pixels, or 10M or more pixels, has been recently put into practical use. Along with an increase in the number of pixels in an image sensor, manufacturers of a digital still camera who uses such a high resolution image sensor as mentioned above is now pursuing maintenance or further improvement of the number of frames per unit time, that is, a speed for continuous photographing. To this end, in a case where the number of pixels is 3M and a speed for continuous photographing is 10 frames per second, for example, a data rate or a rate of readout of image data is 30M pixels per second. Thus, in order to maintain the same speed for continuous photographing, if the number of pixels is 5M, a rate of readout of image data should be 50M pixels per second, if the number of pixels is 8M, a rate of readout of image data should be 80M pixels per second, and if the number of pixels is 10M, a rate of readout of image data should be as high as 100M pixels per second. Nonetheless, the highest operation speed of a CCD sensor is about 35 MHz, and the highest operation speed of a CMOS sensor is in a range from about 50 to 100 MHz. Also, an operation speed of a CMOS LSI is about 100 MHz. In order to form an image sensor at lower costs and with lower power consumption, it is desired that the highest operation speed of an image sensor is about 40 MHz. As such, so long as image data is readout from an image sensor using a single channel, it is impossible to maintain and accomplish a speed for continuous photographing which allows a rate of readout of image data to be 50M pixels per second or higher. In order to radically solve the above-described problem, it is required to increase the number of channels for reading out image data of an image sensor. To meet the foregoing requirement, the recent technologies have achieved implementation of an image sensor including a plurality of readout channels.
As a result of such increase in the number of readout channels as described above, a variety of patterns for reading out image data have been suggested and implemented. Then, a question of how to cope with such diversification of a pattern for reading out image data has been newly raised. In this regard, a brief discussion will be given below by taking a case in which the number of readout channels is two, as an example.
A pattern illustrated in FIGS. 26A to 26E can be thought of as an example of a pattern for reading out (“readout pattern”) based on an RGB Bayer array in the case where the number of readout channels is two. In the readout pattern illustrated in FIGS. 26A to 26E, while only items of image data for R color which are provided in a given even-numbered line of an image sensor are read out via a first readout channel RCH0, only items of image data for Gr color which are provided in the given even-numbered line are read out via a second readout channel RCH1. Then, only items of image data for Gb color which are provided in a given odd-numbered line laid immediately under the given even-numbered line are read out via the first readout channel RCH0, and only items of image data for B color which are provided in the given odd-numbered line are read out via the second readout channel RCH1. Otherwise, in an alternative example, while only items of image data for R color and Gr color which are provided in a given even-numbered line of an image sensor are read out via the first readout channel RCH0, only items of image data for Gb color and B color which are provided in a given odd-numbered line laid immediately under the given even-numbered line are read out by the second readout channel RCH1.
A pattern illustrated in FIGS. 27A, 27B, and 27C can be thought of as another alternative example of a readout pattern based on an RGB Bayer array. Further, as still another alternative example, a readout pattern illustrated in FIG. 28 has been suggested and implemented. In the pattern illustrated in FIG. 28, an RGB Bayer array of an image sensor is divided into a rightward section and a leftward section with a boundary therebetween being set along a center line. Then, items of image data which are provided in a given line are sequentially read out, starting from the leftmost one, and at the same time, the items of image data in the given line are sequentially read out, starting from the rightmost one. Moreover, a readout pattern in which while vertical readout is performed in a given area of an RGB Bayer array of an image sensor, horizontal readout is performed in the remaining area, has been suggested.
As discussed above, such a variety of readout pattern as cited above have been suggested even for the case where the number of readout channels is two. At this rate, in a case where the number of readout channels is more than two, a readout pattern can be further diversified, so that the number of possible readout patterns is enormously increased.
As is made clear from the foregoing discussion, a wide variety of patterns for reading out image data from an image sensor is possible for each of manufacturers of an image sensor, or even for each of models of products offered by the same manufacturer. Accordingly, from a viewpoint of a designer or a manufacturer of a versatile LSI (image processor) for performing image processing on image data read out from an image sensor, there is a need of offering a highly versatile image processing LSI capable of coping with any readout pattern which may be employed in an image sensor utilized by a manufacturer of a digital still camera. To satisfy the foregoing need is a task which should be promptly tackled.
<Second Problem>
An increase in the number of pixels in an image sensor requires further reduction of a size of each of cells in the image sensor because an effective pixel region of the image sensor is unchanged, as described above. Such size reduction of each of cells in an image sensor involves relatively lowering a level of an output signal of each of the cells, to then impair linearity of the output signal of each of the cells. For this reason, a possibility of occurrence of cells having different characteristics depending on a location in an effective pixel region of an image sensor, or depending on a line of an image sensor, is increased. Hence, each of a black level correction value and a white balance correction value which are necessary for normalizing an RGB Bayer signal provided by each of pixels differs from location to location in an effective pixel region of an image sensor. Thus, there is a demand for techniques for correcting an RGB Bayer signal of each of pixels with high accuracy while overcoming the foregoing problem associated with size reduction of each of cells in an image sensor.